1. Field of the Invention
The invention, in general, relates to a method of taking three-dimensional measurements of object surfaces and, more particularly, to a method of using a digital camera for taking such measurements by defining the object coordinates from the curves of the gray scales of images of the object surfaces generated on the sensor of the camera. Such a method is of particular use for measuring static object surfaces.
2. The State of the Art
It is known by means of two cameras to take three-dimensional measurements of object surfaces by triangulation. In such a method, in the initial step a so-called corresponding pair of points is defined in images of the two cameras. These are image points generated by forming images of identical surface points (FIG. 1). If the coordinates of the corresponding pair of points in the two images are known, the triangulation proper may be carried out by calculating the coordinates of the surface point as the intersection of the two xe2x80x9cvisual raysxe2x80x9d.
Points within characteristic image elements of one of the cameras are often chosen for such coordinate calculations. When (automatically) defining a corresponding point in the image of the other camera, it is necessary initially to find the corresponding image element. This may be problematic when there are several similar elements all which may, in theory, apply (FIG. 2). These ambiguities in the definition of corresponding pairs of points (the so-called correspondence problem) have to be solved in order to be able to perform the triangulation. The correspondence or identity problem and, hence, the number of ambiguities, may be substantially reduced by applying epipolar geometry. For reasons of certain geometric necessities the point corresponding to a chosen one can only be positioned on the epipolar line (FIG. 2). However, in order to be able to define the epipolar line, the internal and external camera parameters must be known.
At present, no universal method of solving the correspondence problem is known. There are, however, several methods for solving, or at least significantly reducing, the problem at predetermined conditions of measurements: A widely known method consists of the projection, by a pattern projector, of gray coded patterns (for instance, German patent specifications 3,119,857 A1 and 4,120,115 A1) onto the surface. In this method, very coarsely structured patterns becoming progressively finer are projected for subdividing both images into many corresponding areas. Because of the necessity of multiple projections, this method can only be used in the close-up range and with static objects. Moreover, its costs are of some significance since it requires a relatively high-grade projector suitable for generating precise and, above all, reproducible lines.
In another method, two structurally identical cameras are used the optical axes of which extend parallel to each other. Furthermore, the to image sensors must be positioned in one plane and must be aligned relative to each other at xe2x80x9cthe same levelxe2x80x9d. This is said to be the so-called normal case (Klaus, K.: Photogrammetrie - Grundlagen und Standardverfahren, Band 1, Ferd. Dxc3xcmmlers Verlag, Bonn, 1990) [Klaus, K.: Photogrammetryxe2x80x94Basics and Standard Methods; Vol. 1, Ferd. Dxc3xcmmlers Publisher, Bonn, 1990]. This approach does not solve, but it significantly reduces, the correspondence problem. The described exposure conditions may, however, only be realized by approximation. One example, is the alignment of the optical axes. The principle is as follows: At a relatively small spacing (width of the basis) between the cameras, the difference between corresponding image elements, the so-called parallaxes, are also relatively small. Hence, there is only a small applicable range. The disadvantage of this method is that at a small width of the basis the measurement error is automatically large. If, however, the width of the basis is increased, the areas of possible correspondence are increased as well.
In a further method the image content is analyzed: A search is made for characteristic elements in both images and special parameters for evaluating these elements are calculated (Haralick R. M.; Shapiro, L. G.: Computer and Robot Vision, Volume II, Addison-Welsey Publishing Company, Inc., 1993). This leads to a significant reduction in ambiguities. The method may be used for measuring large objects (e.g. houses) as well as moving objects. The disadvantages of this method are that on the one hand it is not perfect and that, on the other hand, it requires special algorithms which increases the complexity of the calculations. Furthermore, the algorithms may, under certain circumstance, have to be adjusted to the image content.
The examples set forth make clear, as has already been mentioned, that there is as yet no universal solution to the correspondence problem.
It is, therefore, an object of the invention, to propose a method and a related arrangement which for taking three-dimensional optical measurements of object surfaces with a single digital camera provides a simple and substantially certain solution to the correspondence problem. As a peripheral condition, the invention assumes that the surfaces of which measurements are to be taken will act as static objects at least during the time of making exposures.
In accordance with the invention, the object is accomplished by a method of taking three-dimensional measurements of object surfaces in which a digital camera and the object surface sequentially assume at least three different predeterminable positions relative to each other, in which at least one image of the object surface at each of these positions is generated by the digital camera which image is subjected to the operations of:
a) detecting and selecting those elements (edges, corners) or blocks of pixels of sufficient structure in an image generated in the initial position the object coordinates of which are to be defined and of storing the pixel coordinates of the selected elements or pixel blocks;
b) testing in an image taken at an intermediate position between the initial and terminal positions whether in the predetermined close environment of each of the elements or pixel blocks selected in step a) there is present an element of exactly the same kind or a pixel block of sufficient structure of the same kind and, in case of a positive result, validating the elements or pixel blocks, or, otherwise, either excluding the element or pixel block from further processing or repeating at least once the test in respect of an image generated at a position which results in a smaller displacement of the elements or pixel blocks compared to the one of the image previously tested with a negative result;
c) if tracking of elements or pixel blocks considered to have been tracked already, is to be continued for at least a further intermediate position, testing each image generated at these intermediate positions to determine if in the predetermined close proximity of the elements or pixel blocks determined as already tracked in accordance with a previous test there is present exactly one element of the same kind or a pixel block of sufficient structure of the same kind and at a positive result designating the elements or pixel blocks as tracked or, otherwise, excluding the element or pixel block from further processing or of repeating at least once the test for an image generated at a position leading to a smaller displacement of the elements or pixel blocks compared to the displacement of the image previously tested with a negative result;
d) testing in an image generated at a terminal position if in the predetermined close proximity of each elements or pixel block determined by a previous step as having been tracked there is present exactly one element of the same kind or pixel block of the same kind, and in case of a positive result determining these element s or pixel blocks as having been tracked from an initial or terminal position and storing the pixel coordinates of the elements or pixel blocks, and, otherwise, excluding the element or pixel block from further processing or again testing in accordance with step c) an image generated at a position leading to a smaller displacement of the elements or pixel blocks compared to the displacement in the image generated at the terminal position and repeating step d); and
e) calculating the three-dimensional coordinates of the object surfaces in a conventional manner by use of the internal and external camera parameters at the initial and terminal position in respect of the coordination of the elements or pixel blocks determined as having been tracked from the initial to the terminal position.
In another aspect of the invention, the object is accomplished by an arrangement in which the digital camera may be positioned by a mover, motorized or manually, into at least three different predeterminable positions.
The basic concept of the invention resides in the fact that tracking of elements (edges, corners) or of pixel blocks or sufficient structure, from an image generated at the initial position of the camera to an image generated at the terminal position of the camera is made easier and safer when tracking takes place incrementally by way of one or more images generated in camera positions located between the initial and terminal positions of the camera (hereinafter referred to as xe2x80x9cintermediate positionsxe2x80x9d).
Therefore, a particular advantage of the invention resides in the fact that the common compromise between a large and a small base width (distance of the camera between each other) has been solved in an sophisticated manner.
The invention may be utilized with particular advantage whenever it is not possible to use a projector of the kind required in connection with certain prior art methods, for generating a plurality of suitable patterns. This would be the case when taking measurements of large objects (for instance, houses), in particular. However, the invention may also be applied advantageously and cost-efficiently at close ranges as it may be utilized in a cost-efficient and simple manner in view of the fact that no sample projector is required.
A further advantage of the invention resides in the fact that the number of process steps necessary for processing the image data may be adapted to the degree of complication in tracking the elements or pixel blocks.
Other objects will in part be obvious and will in part appear hereinafter.